Annex D
(informative)
A probabilistic approach to determining software
safety integrity for pre-developed software

D.1 General
This annex provides initial guidelines on the use of a probabilistic approach to determining software safety integrity for pre-developed software based on operational experience. This approach is considered particularly appropriate as part of the qualification of operating systems, library modules, compilers and other system software. The annex provides an indication of what is possible, but the techniques should be used only by those who are competent in statistical analysis.

NOTE This annex uses the term confidence level, which is described in IEEE 352. An equivalent term, significance level, is used in IEC 61164.

The techniques could also be used to demonstrate an increase in the safety integrity level of software over time. For example, software built to the requirements of IEC 61508-3 to SIL1 may, after a suitable period of successful operation in a large number of applications, be shown to achieve SIL2.

Table D.1 below shows the number of failure-free demands experienced or hours of failure-free operation needed to qualify for a particular safety integrity level. This table is a summary of the results given in D.2.1 and D.2.3.

Operating experience can be treated mathematically as outlined in D.2 below to supplement or replace statistical testing, and operating experience from several sites may be combined (i.e. by adding the number of treated demands or hours of operation), but only if
– the software version to be used in the E/E/PE safety-related system is identical to the version for which operating experience is being claimed;
– the operational profile of the input space is similar;
– there is an effective system for reporting and documenting failures; and
– the relevant prerequisites (see D.2 below) are satisfied.

D.2 Statistical testing formulae and examples of their use
D.2.1 Simple statistical test for low demand mode of operation
D.2.1.1 Prerequisites
a) Test data distribution equal to distribution for demands during on-line operation.
b) Test runs are statistically independent from each other, with respect to the cause of a failure.
c) An adequate mechanism exists to detect any failures which may occur.
d) Number of test cases n > 100.
e) No failure occurs during the n test cases.

D.2.1.2 Results
Failure probability p (per demand), at the confidence level 1-α, is given by

For a probability of failure on demand of SIL 3 at 95 % confidence the application of the formula gives 30 000 test cases under the conditions of the prerequisites. Table D.1 summarises the results for each safety integrity level.

D.2.2 Testing of an input space (domain) for a low demand mode of operation
D.2.2.1 Prerequisites
The only prerequisite is that the test data is selected to give a random uniform distribution over the input space (domain).

D.2.2.2 Results
The objective is to find the number of tests, n , that are necessary based on the threshold of accuracy, δ , of the inputs for the low demand function (such as a safety shut-down) that is being tested.

D.2.2.3 Example
Consider a safety shut-down that is dependent on just two variables, A and B. If it has been verified that the thresholds that partition the input pair of variables A and B are treated correctly to an accuracy of 1 % of A or B’s measuring range, the number of uniformly distributed test cases required in the space of A and B is